On the Role of Abstraction in Case-Based Reasoning

Cases Conrete Cases level of abstraction high low Abstraction hierarchy C C 1 2 Fig. 2. Abstraction hierarchy for indexing cases This approach to indexing, however, makes an assumption concerning the similarity assessment. It requires that a problem cannot be similar to a concrete case unless it is at least similar to this case at a higher level of abstraction. This assumption holds particularly, if similarity is de ned based on the level of abstraction, which can be done as follows: A problem p is more similar to the concrete case C1 than to the concrete case C2 if the lowest level of abstraction on which p matches C2 is higher (more abstract) than the lowest level of abstraction on which p matches C1. 2.4 Reuse of Abstract Cases There are di erent ways of using the information provided in abstract cases for solving the current problem. No reuse of abstract solutions Abstract cases are only used as indexes to concrete cases. For problem solving, concrete cases are used exclusively. Abstract solutions as result The CBR system retrieves and reuses abstract cases. The abstract solutions contained in the abstract cases are not re ned to more concrete levels but are directly returned as output. The interpretation of abstract solutions is up to the user. Re nement of abstract cases The CBR system retrieves and reuses abstract cases and re nes abstract solutions to the concrete level. The re ned solution is then presented to the user. For her/him it is transparent, whether the solution presented by the system stems directly from a matching concrete cases or whether the solution is obtained through the re nement of an abstract case.solutions as result The CBR system retrieves and reuses abstract cases. The abstract solutions contained in the abstract cases are not re ned to more concrete levels but are directly returned as output. The interpretation of abstract solutions is up to the user. Re nement of abstract cases The CBR system retrieves and reuses abstract cases and re nes abstract solutions to the concrete level. The re ned solution is then presented to the user. For her/him it is transparent, whether the solution presented by the system stems directly from a matching concrete cases or whether the solution is obtained through the re nement of an abstract case. available concrete case new solution 1 new solution 2 (concrete level) Level of Abstraction : 0 1 2 n abstract cases case abstraction refinement Level of Abstraction : Level of Abstraction : Level of Abstraction : Fig. 3. Adaptation by abstraction and re nement Please note that abstraction and re nement is already a technique for solution adaptation (see Figure 3). If available concrete cases are abstracted (e.g., automatically) to abstract cases and then getting retrieved and re ned, a new solution to a new problem will be constructed. The higher the level of abstraction of the reused abstract case, the more may the newly re ned solution di er from the solution contained in the original case. We can distinguish di erent methods for realizing such a re nement: Generative re nement of abstract cases This re nement is done by generative problem solving methods, e.g. hierarchical problem solving. For automatically performing this re nement task, additional general domain knowledge is usually required. Case-based re nement of abstract cases This re nement itself is done in a case-based way, avoiding partially the need for additional general knowledge. However, case-based renement requires cases that describe how the individual elements, the abstract solutions are built of, can be re ned at a more concrete level. 2.5 Adaptation of Abstract Cases Besides the possibility to realize adaptation by re ning abstract cases, adaptation can also be done on a single level of abstraction (see Fig. 4). The spectrum of known methods for solution adaptation in CBR (for an overview see e.g., [8; 14; 34]) can also be applied to abstract cases prior the re nement. Thereby, the exibility of reuse can be increased, i.e., a concrete case covers a larger area in the solution space. available concrete case new solution 1 new solution 2 (concrete level) Level of Abstraction : 0 1 2 n abstract cases case abstraction refinement Level of Abstraction : Level of Abstraction : Level of Abstraction : adaptation adaptation Fig. 4. Adaptation of abstract cases 2.6 Forgetting cases The reuse of cases at several levels of abstraction also provides a frame for realizing case deletion policies [31]. Cases deletion is particularly important to avoid the utility or swamping problem [32; 21; 12] that occurs when case bases grow very large. When reusing abstract cases for indexing and reuse, case deletion can e ciently be realized through a pruning of the abstraction hierarchy. If certain concrete cases are removed from the case base, the abstract cases that remain accessible, can still cover the set of target problems previously covered by the deleted concrete case. However, this requires e ective ways of re nement such as generative or case-based re nement. For selecting cases to forget, the savings due to the reduced retrieval e ort must outweigh the additional e ort for re ning more abstract cases. 2.7 Summary of the Framework The following Table 1 summarizes the various facets of the framework for reusing abstract cases. Existing approaches can be described and analyzed and new approaches can be designed using this framework. kind of s tored cases : abs tract cases for indexing: reuse of abs tract solutions : adaptation of abs tract solutions : case deletion policy: abs tract abs tract & concrete hierarchical abs tract result case based refine. yes no yes yes no no no generative refine. acquis ition of abs tract cases: available manual generation automatic generation cases Table 1. Framework for reusing abstract cases 3 PARIS: Using abstraction in case-based planning Now, we describe a concrete case-based reasoning system, called Paris2 [4; 5; 3] that uses abstraction for case-based planning.Paris can be characterized according to this framework as follows: { abstract and concrete cases are stored in the case base { abstract cases are generated automatically from concrete cases { abstract cases are used for indexing { generative re nement of abstract solutions is realized { adaptation of abstract solutions is possible { case deletion policy is realized. We now explain the approach in more detail. 3.1 Requirements Paris was designed as a generic (i.e., domain independent) case-based planning system but with a particular area of application domains in mind: process planning in mechanical engineering. From this application area, a set of CBR speci c requirements have been identi ed: { ability to cope with vast space of solution plans { construction of correct solutions { exible reuse due to large spectrum of target problems { processing of highly complex cases { only concrete planning cases available (e.g. in archives of a company) 3.2 Abstract Planning Cases We now summarize the formal de nition of concrete and abstract planning cases already presented in detail in [5]. Representation of domains Following a STRIPS-oriented representation [11], a planning domain D de nes a (possibly in nite) set of states S and a set of operators O which describe transitions from one state to a successor state. A state s 2 S is described by a nite set of prepositions. A problem hsI ; sGi in a domain is given by an initial state sI and a goal state sG and a plan (a solution to the problem) is a totally ordered sequence of operators ho1; : : : ; oni that transform the initial state into the goal state. A case is a problem together with a solution to this problem. Di erent levels of abstraction In Paris, di erent levels of abstraction are realized by di erent planning domains. In the following we assume two planning domains: a concrete domain Dc and an abstract domain Da. Concrete and abstract planning domains may represent di erent languages with completely di erent states and operators. A concrete case Cc = hhsc0; scni; (oc1; : : : ; ocn)i is a case given in the concrete planning domain, and an abstract case Ca = hhsa0 ; sami; (oa1; : : : ; oam)i is a case in the abstract planning domain. However, not every abstract case is an abstraction of a concrete case. Additional requirements must be met to call an abstract case abstraction of a concrete case. These requirements can be expressed by the existence of two independent mappings: a state abstraction mapping , and a sequence abstraction mapping [2] (see Fig. 5). 2 Paris stands for plan abstraction and re nement in an integrated system. I 1 2 3 s c s c sc s c s c I s a s a s a s a s G G 1 j i concrete domain: abstract domain: α n i+1 i 4 3 2 1 1 2 m Da Dc α α α j j+1 β(0) = 0 β(1) = 3 β (j) = i β(m) = n Oc Oc Oc Oc Oc Oc Oc Oa Oa Oa Oa Oa Fig. 5. Formalization of case abstraction in planning State Abstraction A state abstraction mapping : Sc ! Sa translates states of the concrete domain into the abstract domain. For this translation, we require additional general domain knowledge about how an abstract state description relates to a concrete state description. We assume that this kind of knowledge can be provided in terms of a domain speci c generic abstraction theory A. Such a generic abstraction theory (expressed by horn clauses) de nes each proposition, abstract states can be composed of, in terms of propositions that can occur in the concrete states. Sequence abstraction The solution to a problem consists of a sequence of operators and a corresponding sequence of states. To relate an abstract solution to a concrete solution, the relationship between the abstract states (or operators) and the concrete states (or operators) must be captured. Each abstract state must have a corresponding concrete state but not every concrete state must have an associated abstract state. The sequence abstraction mapping : N! N selects those states of the concrete problem solution that have a related abstract state. It maps the indices j 2 f1; : : : ;mg of the abstract states saj into the indices i 2 f1; : : : ; ng of the concrete states sci , such that (0) = 0, (m) = n, and (u) < (v) if and only if u < v. This guarantees that abstract and concrete initial and goal states correspond and that the order of states is maintained. Case abstraction Based on the two introduced abstraction functions, our intuition of case abstraction is captured in the following de nition. A case Ca is an abstraction of a case Cc if there exists a state abstraction mapping and a sequence abstraction mapping , such that: saj = (sc (j)) holds for all j 2 f0; : : : ;mg (see Fig. 5). In [5] we have discussed the generality of the presented case abstraction methodology. We have formally shown that hierarchies of abstraction spaces as well as abstractions with respect to di erent aspects can be represented using the presented methodology. Based on the de ned levels of abstraction and the generic abstraction theory, several abstract cases are called abstractions of a single concrete case. 3.3 Acquisition of Abstract Cases Because cases are only available at the concrete domain in the applications domains we have in mind and manual abstraction seems to be a tremendous e ort, abstract cases are generated automatically from a given concrete case. For this purpose, a particular case abstraction algorithm has been developed [5; 3] which could be proven to be correct (computes only correct abstract cases) and complete (computes all abstract cases) with respect to the above introduced model of case abstraction. 3.4 Re nement of abstract cases In Paris an abstract solution contained in an abstract case is re ned automatically to a concrete level solution. If a new target problem is given (at the concrete level), this re nement starts with the concrete initial state from the problem statement (see Fig. 6). A search is performed to nd a sequence of concrete operations which lead to a concrete state that can be abstracted with a state abstraction mapping to match the second abstract state contained in the abstract case. If the rst abstract operator can be re ned a new concrete state is found. This state can then be taken as a starting state to re ne the next abstract operator in the same manner. If this re nement fails we can backtrack to the re nement of the previous operator and try to nd an alternative re nement. If the whole re nement process reaches the nal abstract operator, it must directly search for an operator sequence that leads to the concrete goal state of the new problem. If this concrete goal state has been reached, the concatenation of concrete partial solutions leads to a complete solution to the original problem. 0 1 O 1 O2 Om 2 m abstract case s a sa sa sa α α α α a a a concrete level search spaces ? ? 0 sc n s c n n n n 1 2 3 m Fig. 6. Re nement of abstract cases Abstract solutions decompose the original problem into a set of much smaller subproblems. These subproblems are solved by a search-based problem solver. The problem decomposition leads to a signi cant reduction of the overall search that must be performed to solve the problem [19]. With pure search the worst-case time complexity for nding the required solution is O(bn), where n is the length of the solution and b is the average branching factor3. If the problem is decomposed by an abstract solution into m subproblems, each of which require a solution of length n1; : : : ; nm, respectively, with n1 + n2 + + nm = n, the worst-case time complexity for nding the complete solution is O(bn1 + bn2 + + bnm) which is O(bmax(n1;n2;::: ;nm)). Please note that the re nement e ort increases with the level of abstraction the abstract case is located on, because more abstract solutions contain a smaller number of abstract operators than the detailed solutions do. 3.5 Adaptation of Abstract Cases Paris performs solution adaptation also at a single level of abstraction. For that purpose, an abstract or concrete case is generalized into a generalized case (similar to a schema or script). Such a generalized case does not only describe a single problem and a single solution but a class of problems hsI(xI); sG(xG)i together with a class of solutions 3 The branching factor is the average number of successor states that can be reached through the application of available operators. ho1(x1); : : : ; on(xn)i. Such classes are realized by introducing the variables xj into the initial and goal state as well as into the plan. Additionally, a generalized case contains a set of constraints C(xI; xG; x1; : : : ; xn) that restricts the instantiation of these variables. Paris includes an algorithm for automatically generalizing concrete or abstract cases into schemas (see [3]) by applying explanation-based generalization [22; 9]. This algorithm guarantees the correctness of the computed generalized case, i.e., every instantiation of the occuring variables such that all constraints C(xI; xG; x1; : : : ; xn) are ful lled leads to a correct case, i.e., the plan ho1(x1); : : : ; on(xn)i solves the problem hsI (xI); sG(xG)i Adaptation with generalized cases is done by nding an instantiation of the variables such that hsI(xI); sG(xG)i matches the target problem to be solved and such that the constraints are ful lled. The solution to the target problem results from applying to the solution class of the generalized case. In Paris, matching (similarity assessment) and adaptation is done by a constraint satisfaction problem solver (see [3] for details). The e ort for solving this constraint satisfaction task can be very high: in the worst case it is exponential in the number of constraints and the size of the problem class. Typically, the representations at a higher level of abstraction are less complex than representations at lower levels. Consequently, generalized cases at higher levels of abstraction contain less constraints and the problem class is composed of a small number of prepositions. Therefore, adaptation of abstract cases requires less e ort than adaptation of concrete cases. 3.6 Retrieval with Abstract Cases In Paris abstract and concrete cases are stored in the case base which is organized by an abstraction hierarchy. The idea for constructing this hierarchy is based on the following condition: a case C1 is located above C2 (cf. Fig. 2) if for every problem p holds that if C2 is adaptable for p (at some level of abstraction) then C1 is adaptable for p as well. If this condition can be ful lled, retrieval will be improved, because if C1 is not adaptable for solving p, then none of the cases in the sub-tree below C1 can be adaptable and must consequently not be accessed. Unfortunately, this condition is undecidable in general [3]. However, an abstraction hierarchy can be constructed such that the above condition holds at least for the problems already known (the case base) instead of holding for all possible problems. This approximation approaches the original condition as more and more cases arise. In Paris such an abstraction hierarchy is built and updated incrementally. 3.7 Case Deletion Policy In Paris a utility problem can occur, if the representation (e.g., the concrete domain) is very complex such that matching and adaptation of generalized cases through the constraint propagation becomes very costly. In this case, re ning an abstract case at a higher level of abstraction can involve less e ort than adapting a case at a lower level. To cope with this problem, a case deletion policy is realized which works by pruning sub-trees of the abstraction hierarchy [35]. A sub-tree is pruned if matching and adapting abstract or concrete cases contained in this sub-tree requires more e ort than re ning a more abstract available case. These e orts are estimated through measuring the run-time for matching and adaptation and the run-time for solution re nement based on the problems already contained in the case base. 4 Experimental Evaluation We now present the results of an experimental study on the bene ts of using abstraction in CBR. This study was done using the fully implemented Paris system in the domain of manufacturing planning for rotary symmetric workpieces on a lathe (see [5] for details of the domain). For the experiments, 100 concrete cases were generated randomly. From these concrete cases 28 abstract cases at four levels of abstraction could be generated. 4.1 Improving E ciency of Similarity Assessment and Adaptation The purpose of the rst experiment was to evaluate how the e ort for similarity assessment and adaptation decreases with higher levels of abstraction. For this purpose, we measured the run-time for matching and adapting concrete and abstract generalized cases with the constraint propagation mechanism. A time limit of 200 seconds was imposed for the constraint propagation procedure. If this limit was exceeded, the procedure was terminated and the matching failed. Table 2. Comparison of matching and adapting concrete and abstract cases kind of cases number of constraints run-time in sec. percentage of failures concrete cases 76.3 95.97 42 % abstract cases 21.5 1.11 0 % Table 2 shows for abstract and concrete cases, the average number of constraints to be considered during constraint propagation, the average run-time for matching and adapting, and the percentage of failures due to exceeding of the time limit. As expected, these results show a strong decrease in the run-time for abstract cases which is due to the reduced complexity of abstract cases compared to concrete cases. 4.2 Improving Retrieval by Abstract Cases The purpose of the second experiment is to evaluate the speedup in retrieval time when using abstract cases (organized in an abstraction hierarchy) for indexing. We built up a case-base with 100 concrete and 28 abstract cases. The abstract cases were used as indexes only. We measured the time for retrieving (including matching and adapting) a concrete case with the abstraction hierarchy and compared it to the time for a sequential retrieval of concrete cases. Again, a time limit of 200 seconds was imposed for retrieval. Table 3. Retrieval with abstract cases retrieval method average retrieval time percentage of failures sequential retrieval 185 76 % retrieval with abstract cases 127 58 % Table 3 shows the average results for retrieving 100 di erent cases. We can see a good improvement in the retrieval time as well as a reduction in the number of failures due to exceeding of the time limit. 4.3 Problem Solving Performance The purpose of this experiment is to evaluate the overall problem solving performance and competence for reusing abstract cases vs. reusing concrete cases. From the 100 available cases, we have randomly chosen 10 training sets of 5 cases and 10 training sets of 10 cases. These training sets are selected independently from each other. For each training set, a related testing set is determined by choosing those of the 100 cases which are not used for training. We trained Paris with each of the training sets separately and measured the time for problem solving on the related testing sets. Again, a time-bound of 200 CPU seconds was used for each problem. If the problem could not be solved within this time limit, the problem solver was aborted and the problem remained unsolved. The number of unsolved problems was also evaluated. Table 4. Reuse of abstract cases vs. reuse of concrete cases size of average problem percentage of Reuse method training set solving time unsolved problems 5 56 16% reuse abstract cases 10 49 13 % 5 157 71 % reuse concrete cases 10 154 68 % Table 4 shows the average problem solving time and the average percentage of solved problems for the training sets of the two di erent sizes and the di erent kinds of reuse. These average numbers are computed from the 10 training and testing sets for each size. We can see a strong improvement through reusing abstract case. Additionally, these results were analyzed with the maximally conservative sign test as proposed in [10]. It turned out that in all 20 (10+10) experiments, the improvement was signi cant (p < 0:05). 4.4 Flexibility of Reuse The purpose of this experiment was to evaluate the exibility of the reuse. For each of the 100 cases, we evaluated how many of the problems in the remaining 99 cases could be solved through reuse of this case within the time limit of 200 seconds. We compared the exibility of reusing concrete and abstract cases separately. Figure 7 shows the results plotted for each case. On the abscissa, the 100 cases are ordered according the complexity. Case No. 1 is the simplest case with a plan composed of 4 operators and Case No. 100 is the most complex case containing 18 operators. The ordinate shows the number of problems (of the remaining 99 cases) for which this case can be reused. We can see a strong advantage when reusing abstract cases. The conservative sign test shows the signi cance of this result (p < 0:001). 4.5 Case Deletion Policy Finally we evaluated the impact of the case deletion policy. For that purpose, we trained the system with all available cases and used the same cases for testing it again. In one run, the case deletion policy was active, in the other run it was disabled. Table 5 shows the average problem solving time as well as the number of problems solved within the 200 second time limit. We can identify a signi cant improvement (rank test, p < 0:05) caused by the case deletion policy. 0 10 20 30 40 50 60 70 10 20 30 40 50 60 70 80 90 100 R eu sa b il it y Case number reusing abstract cases reusing concrete cases Fig. 7. Flexibility of reuse Table 5. Case deletion policy Case deletion policy average problem percentage of solving time unsolved problems disabled 69 28 % enabled 36 6 % 4.6 Conclusion from experiments These experiments clearly demonstrate the bene ts we hoped to gain from introducing abstraction into case-based reasoning (cf. section 1). However, these experiments are performed in a very speci c scenario (planning task, domain: process planning for rotary symmetric workpieces, particular representation). Whether these results can be generalized for di erent tasks and domains still has to be proven. However, the results obtained by Branting and Aha [7] { also for a planning task { strongly support our results. 5 Related Work We now discuss related work with respect to the general framework introduced in section 2. We consider the following approaches that reuse cases at several levels of abstraction: { D ej a Vu [29; 30]: design of control software, { PRIAR [16]: domain-independent action planning, { MoCas [24; 4]: model-based case adaptation for diagnosis of technical systems, { COVER and CLOSEST [7]: algorithms for hierarchical A search. Table 6 shows the result of classifying these approaches according to the framework. CLOSEST DEJA-VU kind of s tored cases : abs tract cases for indexing: reuse of abs tract solutions : adaptation of abs tract solutions : case deletion policy: abs tract abs tract & concrete hierarchical